Focused on the issue that the current large-scale networks are not suitable to be applied on resource-starved mobile devices like smart phones and tablet computers, and the pooling layer will lead to the sparsity of feature map, which ultimately affect the recognition accuracy of the neural network, a lightweight face recognition neural network namely ShuffaceNet was proposed, a smooth nonlinear Log-Mean-Exp function ThetaMEX was designed, and an end-to-end trainable ThetaMEX Global Pool Layer (TGPL) was proposed, so as to reduce network parameters and improve computing speed while ensuring the accuracy of the algorithm, achieving the purpose that the network can be effectively deployed on mobile devices with limited resources. ShuffaceNet has about 3 600 parameters, and the model size is only 3.5 MB. The recognition test results on LFW (Labled Faces in the Wild), AgeDB-30 (Age Database-30) and CFP (Celebrities in Frontal Profile) face datasets show that the accuracy of ShuffaceNet reaches 99.32%, 93.17%, 94.51% respectively. Compared with the traditional networks such as MobileNetV1, SqueezeNet and Xception, the proposed network has the size reduced by 73.1%, 82.1% and 78.5% respectively, and the accuracy on AgeDB-30 dataset improved by 5.0%, 6.3% and 6.7% respectively. It can be seen that the proposed network based on ThetaMEX global pooling can improve the model accuracy.